Papers with text-to-text models
ConECT Dataset: Overcoming Data Scarcity in Context-Aware E-Commerce MT (2025.acl-short)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) has improved translation by using Transformer-based models, but still struggles with word ambiguity and context. |
| Approach: | They create a new Czech-to-polish e-commerce product translation dataset coupled with images and product metadata consisting of 11,400 sentence pairs. |
| Outcome: | The proposed model incorporates visual cues alongside textual data to improve translation quality. |
FLIRT: Feedback Loop In-context Red Teaming (2024.emnlp-main)
Copied to clipboard
Ninareh Mehrabi, Palash Goyal, Christophe Dupuy, Qian Hu, Shalini Ghosh, Richard Zemel, Kai-Wei Chang, Aram Galstyan, Rahul Gupta
| Challenge: | Recent work has evaluated the vulnerabilities of large generative models, such as DALL-E, ChatGPT, and GPT-4. |
| Approach: | They propose an automatic red teaming framework that evaluates a given black-box model and exposes its vulnerabilities against unsafe and inappropriate content generation. |
| Outcome: | The proposed framework evaluates a given black-box model and exposes its vulnerabilities against unsafe and inappropriate content generation. |
Cluster-Guided Label Generation in Extreme Multi-Label Classification (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing classification-based models are poorly per-form for tail labels and ignore semantic relations among labels. |
| Approach: | They propose to guide label generation using label cluster information to hierarchically generate lower-level labels. |
| Outcome: | The proposed model outperforms classification and generation baselines on tail labels and improves in four popular XMC benchmarks. |
Extracting Victim Counts from Text (2023.eacl-main)
Copied to clipboard
| Challenge: | Using tagging and regex methods, data on injured, displaced, or abused victims is difficult . data on earthquake injuries and deaths is scarce, subjective, or biased . |
| Approach: | They compare tagging approaches to extract injured, displaced, or abused victims . they discuss calibration and investigate out-of-distribution and few-shot performance . |
| Outcome: | The proposed model is among the first to apply numeracy-focused large language models in a real-world use case with a positive impact. |
Clues Before Answers: Generation-Enhanced Multiple-Choice QA (2022.naacl-main)
Copied to clipboard
| Challenge: | Multiple-choice question answering (MCQA) uses text-to-text framework . but, there is an under-utilization of the decoder and knowledge that can be decoded . |
| Approach: | They propose a generative multiple-choice question answering model which generates a clue from the question and leverages it to enhance a reader for MCQA. |
| Outcome: | The proposed model outperforms text-to-text models on multiple MCQA datasets. |
CSL: A Large-scale Chinese Scientific Literature Dataset (2022.coling-1)
Copied to clipboard
| Challenge: | Existing datasets centered around the English language restrict development of Chinese scientific NLP. |
| Approach: | They present a large-scale Chinese scientific literature dataset based on Chinese papers . they use semi-structured data as a natural annotation for many supervised NLP tasks . |
| Outcome: | The proposed dataset can serve as a Chinese corpus and perform many supervised tasks. |
Evaluation of Transfer Learning for Polish with a Text-to-Text Model (2022.lrec-1)
Copied to clipboard
Aleksandra Chrabrowa, Łukasz Dragan, Karol Grzegorczyk, Dariusz Kajtoch, Mikołaj Koszowski, Robert Mroczkowski, Piotr Rybak
| Challenge: | Recent years have brought significant progress in natural language understanding (NLU) and natural language generation (NLG). |
| Approach: | They propose a benchmark for assessing the quality of text-to-text models for Polish . they evaluate the performance of plT5, mT5, Polish BART, and Polish GPT-2 . |
| Outcome: | The proposed model can be fine-tuned on various NLP tasks with a single training objective. |
FewshotQA: A simple framework for few-shot learning of question answering tasks using pre-trained text-to-text models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained models need fine-tuning on tens of thousands of examples to achieve good results. |
| Approach: | They propose a framework that leverages pre-trained text-to-text models and aligns them with their pre-training framework. |
| Outcome: | The proposed framework outperforms the XLM-Roberta-large on multiple QA benchmarks and is applicable to multilingual situations. |
ReadOnce Transformers: Reusable Representations of Text for Transformers (2021.acl-long)
Copied to clipboard
| Challenge: | ReadOnce Transformers is a task-independent, task-dependent, and compressed representation of text. |
| Approach: | They propose a transformer-based model that can build an information-capturing, task-independent, and compressed representation of text. |
| Outcome: | The proposed model can build an information-capturing, task-independent, and compressed representation of text. |
MedMT5: An Open-Source Multilingual Text-to-Text LLM for the Medical Domain (2024.lrec-main)
Copied to clipboard
Iker García-Ferrero, Rodrigo Agerri, Aitziber Atutxa Salazar, Elena Cabrio, Iker de la Iglesia, Alberto Lavelli, Bernardo Magnini, Benjamin Molinet, Johana Ramirez-Romero, German Rigau, Jose Maria Villa-Gonzalez, Serena Villata, Andrea Zaninello
| Challenge: | Existing studies on large language models for medical applications have focused on a single language . medical mT5 outperforms both encoders and similar sized text-to-text models in English, French, and Italian benchmarks . |
| Approach: | They propose to train Medical mT5, the first open-source text-to-text multilingual model for the medical domain. |
| Outcome: | The proposed model outperforms encoders and similar sized models on the Spanish, French, and Italian benchmarks while being competitive with current state-of-the-art models in English. |